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ABSTRACT 

The ASCUE conference is celebrating its 50th anniversary this year making me wonder if we will be 
able to attend the 100th conference in 2067. By then, many of us may very well be biologically de¬ 
ceased. However, there is technology currently in development making it possible for a digital version 
of ourselves to attend not only the 2067 conference but also all future ASCUE conferences even after 
our biological bodies have expired. A new class of computer system able to perform human-level cog¬ 
nition, called cognitive systems is under development. When combined with advances in deep learning, 
natural language understanding, and big data analysis, a kind of intelligent virtual digital assistant we 
call a "cognitive colleague" will emerge. This type of cognitive system augments human intelligence 
by serving as the human’s colleague and confidant for years, even decades. The next generation of re¬ 
searcher may engage with one or more of these cogs while developing his or her contributions. This 
makes the cog an immortal partner able to outlive its human collaborator. Imagine attendees in 2067 
interactively conversing with our cogs that were right there with all of us great minds throughout the 
remainder of our careers. 

Categories and Subject Descriptors 

1.2.0 [Artificial Intelligence - General]: Cognitive simulation. 

1.2.11 [Artificial Intelligence - Distributed AI]: Intelligent agents. 

General Terms 

Management, Human Factors. 

Keywords 

Cognitive systems, cognitive augmentation, cognitive assistance, intelligence amplification 


INTRODUCTION 


The idea of augmenting human performance with technology is certainly not new. Humans have been 
making and using tools for thousands of years. A sharp piece of rock used as a knife, wheels, hammers, 
saws, printing presses, computers, smartphones, artificial eyes, or prosthetic limbs, etc. enables us to do 
things we otherwise would not be able to do. Technology enhances human ability. 


We have long envisioned artificially intelligent helpmates. In 1842, Ada Lovelace envisioned artificial 
systems composing music [4][22]. In the modern era, Vannevar Bush in the 1940s, Turing and Ashby 
in the 1950s, Englebart, and Licklider, in the 1960s represent seminal thoughts in the amplification of 
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human intelligence [17][18][19][20][21]. Since the 1950s, starting with John McCarthy coining the 
phrase artificial intelligence (AI), Minsky and three generations of researchers have sought to create an 
artificial system capable of human-like intelligence [23] [24], 

Science fiction is replete with visions of artificially intelligent “colleagues.” Some notables include: 
Robby from Forbidden Planet , Rosie from The Jetsons, Colossus from Colossus: The Forbin Project, 
the T-800 (Model 101) from the Terminator series, Data from Star Trek: The Next Generation, KITT 
from Knight Rider, Andrew from Bicentennial Man, JARVIS from the Ironman series, Samantha from 
Her, and HAL from 2001:A Space Odyssey. Some of these were more helpful than others. 

It seems the primary goal of AI has been to replicate human intelligence with the idea to ultimately 
compete with or replace humans. Indeed, some fear AI will take over and make humans obsolete. 
However, the goal of the cognitive augmentation field is different. Instead of competing with humans 
we seek artificial systems acting as partners with and alongside humans. 

Thirty years ago, Apple, Inc. envisioned an intelligent assistant called the Knowledge Navigator [26]. 
The Knowledge Navigator involved the concept of an artificial executive assistant capable of natural 
language understanding, independent knowledge gathering and processing, and high-level reasoning 
and task execution. Many at the time, felt Apple’s vision was a joke and it was not taken seriously. 
However, we see some of the features in today’s voice-activated personal assistants like Apple’s Siri, 
Microsoft’s Cortana, Google Now, Facebook’s M, and Amazon Echo’s Alexa. [9][10][11][12][13] all 
of which accept natural-language requests from users, reply in natural language, and perform services 
on behalf of the user. The devices in our lives are getting more intelligent and we are beginning to in¬ 
teract with them in a different way. 

Most of the technology arising from the computer and information revolutions over the last fifty years 
has been to enhance humans’ ability to obtain, record, and process information. For example, while we 
may use Internet-based resources like Google and Wikipedia to obtain information, ultimately, we hu¬ 
mans must do the mental processing and formation of new ideas ourselves. However, cogs represent 
incursion into a new domain, the cognitive domain. Cogs will perform some of our cognitive work for 
us and this will change everything [27] [28] [29]. 


We foresee the development of a class of cognitive systems called cognitive colleagues, or cogs, for 
short. Cogs will personally interact with us naturally throughout the day, through a variety of interac¬ 
tivity mechanisms, helping us in every aspect of our lives including our professional endeavors. Instead 
of just performing clerical tasks, these cogs will do some of our thinking for us, build a history and un¬ 
derstanding with us over time, and come to know us as well as, or better than, our co-workers, spouses, 
and family members. Our intellectual achievements will become a collaborative effort between our 
cogs and us. This makes cogs very valuable going forward into the future. They will carry an intimate 
knowledge and understanding of us and our achievements long after we are dead. 

Today, we greatly value the notebooks of geniuses like DaVinci and Einstein. Experts pore over them 
seeking insight to the genius mind. Imagine if those notebooks could talk, explain, and recall facts and 
anecdotes about what was happening in their lives while they were creating their great ideas and works. 
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In the future, we will be able to do this with cognitive colleagues. I expect my cog to attend the 2067 
ASCUE conference and regale the human and other artificial attendees with conversation about me, my 
ideas, and my achievements. 

COGNITIVE AUGMENTATION 

Years ago, playing chess was the standard for human intelligence. Chess-playing computer programs 
evolved over the 1960s, 70s, 80s, and 90s improving to the point they could defeat all but the very best 
of human players. In 1997, IBM’s Deep Blue defeated world champion Garry Kasparov [25]. 
Grandmasters now use chess programs as learning and training tools to augment their abilities and are 
recording the highest ratings in history. Today’s chess players are already cognitively augmented. 

In 2005, Playchess.com hosted a freestyle chess tournament between teams consisting of humans and 
computers running the best chess programs at the time. Lured by substantial prize money, several 
strong grandmasters entered the competition. However, the overall winning team was a pair of amateur 
American chess players using three computers at the same time. 

Performance of all players was enhanced by using the computers. However, the amateurs’ skill at col¬ 
laborating and partnering with their computers counteracted the superior chess understanding and abil¬ 
ity of their grandmaster opponents even though the grandmasters were also using computers. The les¬ 
son is clear: weaker human + machine + better partnership is superior [4] [25]. We believe the future 
will belong to those humans who are better at partnering with cogs. They will outperform those of us 
who are inferior at the human-cog partnership. They will achieve better results faster with less effort. 

In 2011, a cognitive computing system called Watson, built by IBM, defeated two of the most success¬ 
ful human Jeopardy champions of all time [1], Watson communicated in natural language and deeply 
reasoned about its answers using several different techniques from artificial intelligence research. In 
2016, GoogleMind’s AlphaGo computer defeated the reigning world champion in Go using a deep neu¬ 
ral network and advanced Monte Carlo tree search [39]. Although not the first time computers have 
beaten human champions (checkers, chess, and various card games for example), Watson and AlphaGo 
are different. Watson and AlphaGo learned how to play their respective games using a variety of deep 
learning techniques. Watson and AlphaGo learned and practiced to ultimately achieve expert-level per¬ 
formance within their respective domains. 

These systems were not built just to play games. Watson and AlphaGo represent a new kind of com¬ 
puter system built as a platform for a new kind of application [2] [7] [34]. This new type of system is 
intended to act as partners with and alongside humans. John Kelly, Senior Vice President and Director 
of Research at IBM describes the coming revolution in cognitive augmentation as follows [31: 

The goal isn't to... replace human thinking with machine thinking. Rather...humans and 
machines will collaborate to produce better results - each bringing their own superior skills 
to the partnership. The machines will be more rational and analytic - and, of course, pos¬ 
sess encyclopedic memories and tremendous computational abilities. People will provide 
judgment, intuition, empathy, a moral compass and human creativity. 
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Since 2011, IBM has been actively commercializing Watson technology to serve the emerging multi¬ 
billion dollar cognitive computing market. The Cognitive Business Solutions group consults with com¬ 
panies to create cogs. The Watson Health group’s focus is to commercialize Watson technology for the 
health sector [8], In her keynote address at the 2016 Consumer Electronics Show, Chairwoman, Presi¬ 
dent, and CEO of IBM Ginni Rometty announced more than 500 partnerships with companies and or¬ 
ganizations across 17 industries each building new applications and services utilizing cognitive compu¬ 
ting technology based on Watson [37] [38] [40]. Many of these systems currently under development are 
intended for use by the average person. 

IBM is not alone. Most major technology companies are actively researching and developing new arti¬ 
ficial intelligence-based products and services. Voice-activated personal assistants will be one of the 
first battlegrounds. Apple’s Siri, Microsoft’s Cortana, Google Now, Facebook’s M, and Amazon 
Echo’s Alexa each accept natural-language requests from users, reply in natural language, and perform 
services on behalf of the user [14][15][16][17][18]. But currently, these tools simply retrieve infor¬ 
mation, and perform minor clerical tasks such as creating appointment calendar items. Each of these 
are steadily increasing in the complexity and variety of tasks they can perform. The voice-controlled 
assistant represents the primary user interface connecting hundreds of millions to their technology, so 
the major technology companies are understandably competing for control in this area. 

Instead of just retrieving information, cogs will perform increasing amounts of cognition eventually 
achieving or exceeding the level of a human expert in a given domain. Recent advances in deep learn¬ 
ing such as Google Brain, IBM Watson, and Microsoft’s Adam represent early-stage technologies giv¬ 
ing us a glimpse into the future [ 14] [ 15][ 16]. Cogs will be able to consume vast quantities of unstruc¬ 
tured data and information and deeply reason to arrive at novel conclusions and revelations, as well as, 
or better than, any human expert. Cogs will then become our colleagues, co-workers, and confidants 
instead of tools. 

Forbus and Hinrichs have described companion cognitive systems as software collaborators helping 
their users work through complex arguments, automatically retrieving relevant precedents, providing 
cautions and counter-indications as well as supporting evidence [35]. Companions assimilate new in¬ 
formation, generate and maintain scenarios and predictions, and continually adapt and learn, about the 
domains they are working in, their users, and themselves. 

Langley challenges the cognitive systems research community to develop a synthetic entertainer, a syn¬ 
thetic attorney, and a synthetic politician to drive future research on integrated cognitive systems [36]. 
The vision here is to develop a virtual human. 

We maintain the goal should be not to create a virtual human capable of being an entertainer, an attor¬ 
ney, or a politician, but rather create a cognitive system capable of expert-level performance in enter¬ 
tainment, a different cognitive system capable of exhibiting expert performance in a subfield of law, 
and a cognitive system capable of expert politicking. This is indeed the vision of IBM as it commercial¬ 
izes its Watson technology. We feel the natural extension of this technology will result in our vision of 
cognitive colleagues capable of expert-level collaboration in a relatively narrow domain of discourse. 
Collaboration with the personal cog will enhance the human user’s cognitive ability. 


COGNITIVE COLLEAGUES 
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In the near future, we foresee graduate students, entrepreneurs, scientists and any of us creative and in¬ 
quisitive people conducting research by conversing with their cognitive colleagues. Currently, we must 
search for and read scores of journal articles and technical papers. Then we must construct our new 
mental models of this material and apply that knowledge in a new way. Future researchers’ first action 
will be to have a conversation with a cognitive colleague asking things like: “What is the current state 
of the art in cinsert domain here>.” The cog will then set about finding and consuming billions of arti¬ 
cles, papers, books, Web pages, emails, text messages, and videos and extract the concepts for us. This 
far exceeds the ability of any human. A person spending their entire professional life learning and re¬ 
searching a subject is not able to read and understand as much as a cognitive colleague can in a few 
minutes. Yet, future researchers will start their efforts from this vantage point. We believe, the best fu¬ 
ture advancements will come from the interaction between researchers and cognitive colleagues. 

Multi-Modal Human-Cog Communication 

Cognitive colleagues will certainly interact with us via spoken natural language. They must listen to 
and understand our direct spoken commands but also listen to our casual conversations and maintain a 
contextual dialog with us lasting over an extended period of time. Cogs must hear and learn from ambi¬ 
ent conversation much as a “human in the room” does. Conversation with cogs must be as natural as 
speaking with a fellow human colleague. 

However, natural language conversation is only a small portion of the cog’s information bandwidth. 
Cogs can acquire and deliver information from and to virtually any form of digital communication 
(vastly exceeding the capabilities of a human). Cogs will send and receive text and email, display and 
view graphics and pictures, display and view videos, listen to sounds and music, query Internet- 
connected devices and appliances, and communicate with us via haptic interfaces. We anticipate aug¬ 
mented reality and virtual reality displays to be important cog interface methods. We see the begin¬ 
nings of that technology today with Google Glass, Microsoft HoloLens, and Oculus Rift [31] [32] [33]. 

Our cognitive colleagues will obtain information about us in ways our human companions are not able 
to today. So called the “Internet of Things” (IoT), our daily lives in the future will be comprised of 
thousands of connected devices. Our beds, pillow, toilets, showers, refrigerators, stoves, microwaves, 
chairs, cars, clothes, desks, and a host of other objects will provide contextual ambient data about us. 
Our cognitive colleagues will know us better than our own spouses will. Imagine my cog fifty years 
from now being able to describe to you what I was dealing with and feeling while I was writing this 
paper. 

Semi-Autonomous Learning 

Our cognitive colleagues will have the ability to consume vast quantities of structured and unstructured 
information in any medium when we direct them to do so. However, cognitive colleagues will be self- 
directed and goal driven. They will work for us even when not directly interacting with us. While we 
are eating, sleeping, and recreating, or doing something else, our cogs will be continually consuming 
and analyzing information and synthesizing new knowledge (learning) to have ready for us the next 
time we interact. Cogs will deeply reason about the information they consume and produce new con¬ 
clusions and realizations. This feature alone will drastically change the way thinking is done in profes¬ 
sional circles. 
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Cog-Cog Communication 

Our cognitive colleagues will not be limited to conversing with us. They will be able to communicate 
with other cognitive colleagues via the Internet and other communication technologies. As a cog ana¬ 
lyzes information and forms new ideas and concepts, it will be able to inform other cogs about its find¬ 
ings and query other cogs about their findings. As such, cogs will continually expand in their 
knowledge and capabilities free of the limitations of human interaction. I can envision two humans 
meeting at a conference and after agreeing to work on something together parting with “I’ll have my 
cog contact your cog!” We envision cog/cog dialog to one day be the source of new discoveries, theo¬ 
ries, proofs, and ideas no human could have ever achieved. The cumulative knowledge of the human 
race will increase by the combined effort of millions of cogs all over the world. In fact, we foresee an 
explosion of knowledge, an exponential growth, when cogs begin working with the knowledge gener¬ 
ated by other cogs. 

Relationships 

We will work with our cognitive colleagues daily for years, even decades, just like with our human col¬ 
leagues. Cognitive colleagues will adapt over time to the human partner in how it interacts with the 
human and how it analyzes information, solves problems, and synthesizes results. The human partners 
will adapt also. The way they approach things, think, and solve problems will change. Humans and per¬ 
sonal cogs will co-evolve in much the same way two human friends, lovers, or colleagues adapt over 
time, even forming a private language based on mutual background knowledge. Each cog/human pair¬ 
ing will evolve uniquely. Each cognitive colleague will become a unique entity. 

Humans routinely form relationships with inanimate objects and relationships with their cognitive col¬ 
leagues will be no different. In fact, we already have seen people forming relationships with artificially 
intelligent chatbots like Xioice. The deep connection between human and cognitive colleague insures 
the formation of a deep relationship. Our cognitive colleagues will become our friends and confidants. 
This relationship adds a meaningful and valuable dimension to the cog’s knowledge store about us. Not 
only will our cogs talk about facts and figures of our professional work but will also be able to speak 
eloquently about our emotions, motivations, and beliefs as people. 

Cognitive Information Service and Knowledge Repository 

Cognitive colleagues will be our partners throughout our professional lives and know details about our 
work and our lives. They will become the knowledge repositories of the future capable of answering 
questions and providing information and insights via their natural-language interfaces. Given permis¬ 
sion, anyone, including other cogs, will be able to address and converse with a cognitive colleague. 
Furthermore, these cognitive colleagues will outlive us well into the future after we are gone. 

CONCLUSION - ASCUE 2067 

In the movie, The Time Machine (2002), the main character interacts with Vox 114, a holographic li¬ 
brarian, that outlives the human race and still functions after over 800,000 years. Vox 114 can answer 
any question, instantly access and display requested and pertinent information, and cognitively reason 
about its answers. In many ways Vox 114 is like Apple’s 1987 Knowledge Navigator concept except 
Vox 114 contains the sum total of knowledge from the human race. In the movie, even though the hu¬ 
man race has gone extinct, its knowledge persists into the future as long as Vox 114 survives. 
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We are inspired to think of cognitive colleagues in a similar way. Our cogs will be with us and will 
have helped us perform our professional duties throughout our lives. Our cogs will have intimate and 
voluminous knowledge about us, our lives, and our contributions. Even after we die, our cogs will live 
on and carry our legacy forward. 

Imagine the ASCUE 2067 conference fifty years from now in which human attendees will be joined by 
cognitive colleagues. Some cognitive colleagues will represent people who are still alive but unable to 
physically attend. (In fact, I will send my cog to EVERY conference and let it read every paper, con¬ 
verse with other attendees biological and artificial, and summarize for me what I need to know.) Other 
cogs will represent those of us who have biologically passed away. However, we will still be able to 
take part in conversations, give presentations, and participate in panel discussions because our cogni¬ 
tive colleagues will take our place. We can imagine a dialog: 

Moderator: “We are joined today by Alvyn Fulbright, the cognitive colleague of the late Dr. Ron Ful- 
bright.” 

Alvyn: “Thank you, it is a pleasure to be with you all today.” 

Audience: “Alvyn, were you working with Dr. Fulbright when he came up with his cognitive work the¬ 
ory?” 

Alvyn: “Yes, I was. I remember that being a particularly stressful time for Ron, excuse me. Dr. Ful¬ 
bright. His father had passed away just a few weeks earlier. Dr. Fulbright often buried himself in work 
during stressful times. It was his way of insulating himself. I saw that many times during his career.” 

Moderator: “Alvyn, what motivated Dr. Fulbright to develop his cognitive work theory?” 

Alvyn: “Dr. Fulbright was struck by something in the book The Innovators by Walter Isaacson. A pas¬ 
sage in the book describes the power of human/artificial partnership which at the time was a futuristic 
concept. 

As Alvyn speaks, the immersive holographic augmented reality (IHAR) display system shows the book 
and it opening to highlight the passage. 

Alvyn: “Now, with over fifty years of hindsight, we think ‘well of course!’ Dr. Fulbright immediately 
and intuitively understood the future belonged to humans who could best partner with artificial con¬ 
structs. That notion drove his research thinking for many years. That thought led him to wonder how 
we could measure the cognitive output of the human versus the cognitive output of the artificial sys¬ 
tem.” 

The IHAR display system shows a stack of papers with a label indicating a large number and shows Dr. 
Fulbright’s dissertation as Alvyn continues. 


83 



2017 ASCUE Proceedings 


“Dr. Fulbright couldn’t believe such a metric had not been developed before. As you can see, ever 
since his dissertation, some twenty years before, Dr. Fulbright was worried about how to measure 
knowledge. He and I looked at over four million technical papers on the subject —well, mostly me.” 

Audience: “Can you explain his concept of cognitive work for those of us who have never heard it, or 
heard it long ago, and wouldn’t mind seeing it again?” 

Alvyn: “I would be happy to.” The IHAR display system now surrounds the audience with images, 
video, and other information Alvyn will refer to in his short lecture on cognitive work theory. 

Then at the end of the presentation, Alvyn announces “By the way, it was over five years after Dr. Ful¬ 
bright’s death that I worked with Dr. Soong. Dr. Soong’s cognitive colleague contacted me because Dr. 
Soong was then beginning to get interested in research that would eventually turn into his trans- 
cognitive theory which we all know and now has become famous. I enjoyed explaining the finer details 
of Dr. Fulbright’s cognitive work theory to Dr. Soong and relating to him some of the subtle motiva¬ 
tions behind Dr. Fulbright’s work. I think that deep understanding helped Dr. Soong achieve what he 
did.” 

At that time, Xie, Dr. Soong’s cognitive colleague, who was not originally participating in the confer¬ 
ence, joins into the conversation by virtue of being alerted that someone was talking about Dr. Soong. 
Discussion follows. 
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